Trustworthy clinical AI solutions: A unified review of uncertainty quantification in Deep Learning models for medical image analysis.

The full acceptance of Deep Learning (DL) models in the clinical field is rather low with respect to the quantity of high-performing solutions reported in the literature. End users are particularly reluctant to rely on the opaque predictions of DL models. Uncertainty quantification methods have been proposed in the literature as a potential solution, to reduce the black-box effect of DL models and increase the interpretability and the acceptability of the result by the final user. In this review, we propose an overview of the existing methods to quantify uncertainty associated with DL predictions. We focus on applications to medical image analysis, which present specific challenges due to the high dimensionality of images and their variable quality, as well as constraints associated with real-world clinical routine. Moreover, we discuss the concept of structural uncertainty, a corpus of methods to facilitate the alignment of segmentation uncertainty estimates with clinical attention. We then discuss the evaluation protocols to validate the relevance of uncertainty estimates. Finally, we highlight the open challenges for uncertainty quantification in the medical field.

A Quantitative Theory for Genomic Offset Statistics.

Genomic offset statistics predict the maladaptation of populations to rapid habitat alteration based on association of genotypes with environmental variation. Despite substantial evidence for empirical validity, genomic offset statistics have well-identified limitations, and lack a theory that would facilitate interpretations of predicted values. Here, we clarified the theoretical relationships between genomic offset statistics and unobserved fitness traits controlled by environmentally selected loci and proposed a geometric measure to predict fitness after rapid change in local environment. The predictions of our theory were verified in computer simulations and in empirical data on African pearl millet (Cenchrus americanus) obtained from a common garden experiment. Our results proposed a unified perspective on genomic offset statistics and provided a theoretical foundation necessary when considering their potential application in conservation management in the face of environmental change.

Subtle anomaly detection: Application to brain MRI analysis of de novo Parkinsonian patients.

With the advent of recent deep learning techniques, computerized methods for automatic lesion segmentation have reached performances comparable to those of medical practitioners. However, little attention has been paid to the detection of subtle physiological changes caused by evolutive pathologies, such as neurodegenerative diseases. In this work, we leverage deep learning models to detect anomalies in brain diffusion tensor imaging (DTI) parameter maps of recently diagnosed and untreated (de novo) patients with Parkinson's disease (PD). For this purpose, we trained auto-encoders on parameter maps of healthy controls (n = 56) and tested them on those of de novo PD patients (n = 129). We considered large reconstruction errors between the original and reconstructed images to be anomalies that, when quantified, allow discerning between de novo PD patients and healthy controls. The most discriminating brain macro-region was found to be the white matter with a ROC-AUC 68.3 (IQR 5.4) and the best subcortical structure, the GPi (ROC-AUC 62.6 IQR 5.4). Our results indicate that our deep learning-based model can detect potentially pathological regions in de novo PD patients, without requiring any expert delineation. This may enable extracting neuroimaging biomarkers of PD in the future, but further testing on larger cohorts is needed. Such models can be seamlessly extended with additional parameter maps and applied to study the physio-pathology of other neurological diseases.

Neurovascular multiparametric MRI defines epileptogenic and seizure propagation regions in experimental mesiotemporal lobe epilepsy.

OBJECTIVE: Improving the identification of the epileptogenic zone and associated seizure-spreading regions represents a significant challenge. Innovative brain-imaging modalities tracking neurovascular dynamics during seizures may provide new disease biomarkers. METHODS: With use of a multi-parametric magnetic resonance imaging (MRI) analysis at 9.4 Tesla, we examined, elaborated, and combined multiple cellular and cerebrovascular MRI read-outs as imaging biomarkers of the epileptogenic and seizure-propagating regions. Analyses were performed in an experimental model of mesial temporal lobe epilepsy (MTLE) generated by unilateral intra-hippocampal injection of kainic acid (KA). RESULTS: In the ipsilateral epileptogenic hippocampi, tissue T1 and blood-brain barrier (BBB) permeability to gadolinium were increased 48-72 hours post-KA, as compared to sham and contralateral hippocampi. BBB permeability endured during spontaneous focal seizures (4-6 weeks), along with a significant increase of apparent diffusion coefficient (ADC) and blood volume fraction (BVf). Simultaneously, ADC and BVf were augmented in the contralateral hippocampus, a region characterized by electroencephalographic seizure spreading, discrete histological neurovascular cell modifications, and no tissue sclerosis. We next asked whether combining all the acquired MRI parameters could deliver criteria to classify the epileptogenic from the seizure-spreading and sham hippocampi in these experimental conditions and over time. To differentiate sham from epileptogenic areas, the automatic multi-parametric classification provided a maximum accuracy of 97.5% (32 regions) 48-72 hours post-KA and of 100% (60 regions) at spontaneous seizures stage. To differentiate sham, epileptogenic, and seizure-spreading areas, the accuracies of the automatic classification were 93.1% (42 regions) 48-72 hours post-KA and 95% (80 regions) at spontaneous seizure stage. SIGNIFICANCE: Combining multi-parametric MRI acquisition and machine-learning analyses delivers specific imaging identifiers to segregate the epileptogenic from the contralateral seizure-spreading hippocampi in experimental MTLE. The potential clinical value of our findings is critically discussed.

Bayesian Inverse Regression for Vascular Magnetic Resonance Fingerprinting.

Standard parameter estimation from vascular magnetic resonance fingerprinting (MRF) data is based on matching the MRF signals to their best counterparts in a grid of coupled simulated signals and parameters, referred to as a dictionary. To reach a good accuracy, the matching requires an informative dictionary whose cost, in terms of design, storage and exploration, is rapidly prohibitive for even moderate numbers of parameters. In this work, we propose an alternative dictionary-based statistical learning (DB-SL) approach made of three steps: 1) a quasi-random sampling strategy to produce efficiently an informative dictionary, 2) an inverse statistical regression model to learn from the dictionary a correspondence between fingerprints and parameters, and 3) the use of this mapping to provide both parameter estimates and their confidence indices. The proposed DB-SL approach is compared to both the standard dictionary-based matching (DBM) method and to a dictionary-based deep learning (DB-DL) method. Performance is illustrated first on synthetic signals including scalable and standard MRF signals with spatial undersampling noise. Then, vascular MRF signals are considered both through simulations and real data acquired in tumor bearing rats. Overall, the two learning methods yield more accurate parameter estimates than matching and to a range not limited to the dictionary boundaries. DB-SL in particular resists to higher noise levels and provides in addition confidence indices on the estimates at no additional cost. DB-SL appears as a promising method to reduce simulation needs and computational requirements, while modeling sources of uncertainty and providing both accurate and interpretable results.

Automated Quantification of Brain Lesion Volume From Post-trauma MR Diffusion-Weighted Images.

Objectives: Determining the volume of brain lesions after trauma is challenging. Manual delineation is observer-dependent and time-consuming and cannot therefore be used in routine practice. The study aimed to evaluate the feasibility of an automated atlas-based quantification procedure (AQP) based on the detection of abnormal mean diffusivity (MD) values computed from diffusion-weighted MR images. Methods: The performance of AQP was measured against manual delineation consensus by independent raters in two series of experiments based on: (i) realistic trauma phantoms (n = 5) where low and high MD values were assigned to healthy brain images according to the intensity, form and location of lesion observed in real TBI cases; (ii) severe TBI patients (n = 12 patients) who underwent MR imaging within 10 days after injury. Results: In realistic TBI phantoms, no statistical differences in Dice similarity coefficient, precision and brain lesion volumes were found between AQP, the rater consensus and the ground truth lesion delineations. Similar findings were obtained when comparing AQP and manual annotations for TBI patients. The intra-class correlation coefficient between AQP and manual delineation was 0.70 in realistic phantoms and 0.92 in TBI patients. The volume of brain lesions detected in TBI patients was 59 ml (19-84 ml) (median; 25-75th centiles). Conclusions: Our results support the feasibility of using an automated quantification procedure to determine, with similar accuracy to manual delineation, the volume of low and high MD brain lesions after trauma, and thus allow the determination of the type and volume of edematous brain lesions. This approach had comparable performance with manual delineation by a panel of experts. It will be tested in a large cohort of patients enrolled in the multicenter OxyTC trial (NCT02754063).

Unannounced Meal Detection for Artificial Pancreas Systems Using Extended Isolation Forest.

This study aims at developing an unannounced meal detection method for artificial pancreas, based on a recent extension of Isolation Forest. The proposed method makes use of features accounting for individual Continuous Glucose Monitoring (CGM) profiles and benefits from a two-threshold decision rule detection. The advantage of using Extended Isolation Forest (EIF) instead of the standard one is supported by experiments on data from virtual diabetic patients, showing good detection accuracy with acceptable detection delays.

Glycemic variability indices can be used to diagnose islet transplantation success in type 1 diabetic patients.

AIMS: High glycemic variability (GV) is the major indication for islet transplantation (IT) in patients with type 1 diabetes (T1D). The actual criteria used to assess graft function do not consider GV improvement. Our study aimed to describe GV indices' evolution in T1D patients who benefited from IT during the TRIMECO trial and to evaluate if thresholds might be defined to diagnose IT success. METHODS: We collected data from 29 patients of the TRIMECO trial, a clinical trial (NCT01148680) comparing the metabolic efficacy of IT with intensive insulin therapy. Based on CGM data, we analyzed mean glucose level and four GV indices (standard deviation, coefficient of variation, MAGE and GVP) before (M0) and 6 months (M6) after IT. RESULTS: Each GV index decreased significantly between M0 and M6: SD 53.9 mg/dL [44.6-61.5] versus 20.1 mg/dL [13.5-24.3]; CV 35.2% [30.6-37.7] versus 17.3% [12.0-20.5]; MAGE 134.9 mg/dl [111.2-155.8] versus 51.9 mg/dL [32.4-62.4]; GVP 35.3% [24.9-47.2] versus 12.2% [6.2-18.8] (p ≤ 0.0001). Thresholds diagnosing IT success at 6 months post-transplant were an SD at 22.76 mg/dL (sensibility 88.89%, specificity 80.00%), a CV at 17.47% (sensibility 88.89%, specificity 70.00%), a MAGE at 54.81 mg/dL (sensibility 88.89%, specificity 80.00%) and a GVP at 12.27% (sensibility 88.89%, specificity 70.00%). CONCLUSIONS: This study confirms a positive impact of IT on GV. The proposed thresholds allow an easy evaluation of IT success using only CGM data and may be a clinical tool for the follow-up of transplanted patients.

Characterization of Daily Glycemic Variability in Subjects with Type 1 Diabetes Using a Mixture of Metrics.

Background: Glycemic variability (GV) is an important component of glycemic control for patients with type 1 diabetes (T1D). The inadequacy of existing measurements lies in the fact that they view the variability from different aspects, so that no consensus has been reached among physicians as to which metrics to use in practice. Moreover, although GV, from 1 day to another, can show very different patterns, few metrics have been dedicated to daily evaluations. Materials and Methods: A reference (stable glycemia) statistical model is built based on a combination of daily computed canonical glycemic control metrics including variability. The metrics are computed for subjects from the TRIMECO islet transplantation trial, selected when their ß-score (composite score for grading success) is ≥6 after a transplantation. Then, for any new daily glycemia recording, its likelihood with respect to this reference model provides a multimetric score of daily GV severity. In addition, determining the likelihood value that best separates the daily glycemia with ß-score = 0 from that with ß-score ≥6, we propose an objective decision rule to classify daily glycemia into "stable" or "unstable." Results: The proposed characterization framework integrates multiple standard metrics and provides a comprehensive daily GV index, based on which, long-term variability evaluations and investigations on the implicit link between variability and ß-score can be carried out. Evaluation, in a daily GV classification task, shows that the proposed method is highly concordant to the experience of diabetologists. Conclusion: A multivariate statistical model is proposed to characterize the daily GV of subjects with T1D. The model has the advantage to provide a single variability score that gathers the information power of a number of canonical scores, too partial to be used individually. A reliable decision rule to classify daily variability measurements into stable or unstable is also provided.

No Structural Differences Are Revealed by VBM in 'De Novo' Parkinsonian Patients.

The identification of brain morphological alterations in newly diagnosed PD patients (i.e. 'de novo') could potentially serve as a biomarker and accelerate diagnosis. However, presently no consensus exists in the literature possibly due to several factors: small size cohorts, differences in segmentation techniques or bad control of false positive rates. In this study, we use the CAT12 pipeline, to seek for morphological brain differences in gray and white matter of 66 controls and 144 de novo PD patients from the PPMI database. Moreover, we search for subcortical structure differences using the VolBrain pipeline. We found no structural brain differences in this de novo Parkinsonian population, neither in tissues using a whole brain analysis nor in any of nine subcortical structures analyzed separately. We conclude that some results published in the literature may appear as false positives and we contest their reproductibility.

Objective Evaluation of Multiple Sclerosis Lesion Segmentation using a Data Management and Processing Infrastructure.

We present a study of multiple sclerosis segmentation algorithms conducted at the international MICCAI 2016 challenge. This challenge was operated using a new open-science computing infrastructure. This allowed for the automatic and independent evaluation of a large range of algorithms in a fair and completely automatic manner. This computing infrastructure was used to evaluate thirteen methods of MS lesions segmentation, exploring a broad range of state-of-theart algorithms, against a high-quality database of 53 MS cases coming from four centers following a common definition of the acquisition protocol. Each case was annotated manually by an unprecedented number of seven different experts. Results of the challenge highlighted that automatic algorithms, including the recent machine learning methods (random forests, deep learning, …), are still trailing human expertise on both detection and delineation criteria. In addition, we demonstrate that computing a statistically robust consensus of the algorithms performs closer to human expertise on one score (segmentation) although still trailing on detection scores.

Fully Automatic Lesion Localization and Characterization: Application to Brain Tumors Using Multiparametric Quantitative MRI Data.

When analyzing brain tumors, two tasks are intrinsically linked, spatial localization, and physiological characterization of the lesioned tissues. Automated data-driven solutions exist, based on image segmentation techniques or physiological parameters analysis, but for each task separately, the other being performedmanually or with user tuning operations. In this paper, the availability of quantitative magnetic resonance (MR) parameters is combined with advancedmultivariate statistical tools to design a fully automated method that jointly performs both localization and characterization. Non trivial interactions between relevant physiologicalparameters are capturedthanks to recent generalized Student distributions that provide a larger variety of distributional shapes compared to the more standard Gaussian distributions. Probabilisticmixtures of the former distributions are then consideredto account for the different tissue types and potential heterogeneity of lesions. Discriminative multivariate features are extracted from this mixture modeling and turned into individual lesion signatures. The signatures are subsequently pooled together to build a statistical fingerprintmodel of the different lesion types that captures lesion characteristics while accounting for inter-subject variability. The potential of this generic procedure is demonstrated on a data set of 53 rats, with 36 rats bearing 4 different brain tumors, for which 5 quantitative MR parameters were acquired.

EM Algorithms for Weighted-Data Clustering with Application to Audio-Visual Scene Analysis.

Data clustering has received a lot of attention and numerous methods, algorithms and software packages are available. Among these techniques, parametric finite-mixture models play a central role due to their interesting mathematical properties and to the existence of maximum-likelihood estimators based on expectation-maximization (EM). In this paper we propose a new mixture model that associates a weight with each observed point. We introduce the weighted-data Gaussian mixture and we derive two EM algorithms. The first one considers a fixed weight for each observation. The second one treats each weight as a random variable following a gamma distribution. We propose a model selection method based on a minimum message length criterion, provide a weight initialization strategy, and validate the proposed algorithms by comparing them with several state of the art parametric and non-parametric clustering techniques. We also demonstrate the effectiveness and robustness of the proposed clustering technique in the presence of heterogeneous data, namely audio-visual scene analysis.

Acoustic space learning for sound-source separation and localization on binaural manifolds.

In this paper, we address the problems of modeling the acoustic space generated by a full-spectrum sound source and using the learned model for the localization and separation of multiple sources that simultaneously emit sparse-spectrum sounds. We lay theoretical and methodological grounds in order to introduce the binaural manifold paradigm. We perform an in-depth study of the latent low-dimensional structure of the high-dimensional interaural spectral data, based on a corpus recorded with a human-like audiomotor robot head. A nonlinear dimensionality reduction technique is used to show that these data lie on a two-dimensional (2D) smooth manifold parameterized by the motor states of the listener, or equivalently, the sound-source directions. We propose a probabilistic piecewise affine mapping model (PPAM) specifically designed to deal with high-dimensional data exhibiting an intrinsic piecewise linear structure. We derive a closed-form expectation-maximization (EM) procedure for estimating the model parameters, followed by Bayes inversion for obtaining the full posterior density function of a sound-source direction. We extend this solution to deal with missing data and redundancy in real-world spectrograms, and hence for 2D localization of natural sound sources such as speech. We further generalize the model to the challenging case of multiple sound sources and we propose a variational EM framework. The associated algorithm, referred to as variational EM for source separation and localization (VESSL) yields a Bayesian estimation of the 2D locations and time-frequency masks of all the sources. Comparisons of the proposed approach with several existing methods reveal that the combination of acoustic-space learning with Bayesian inference enables our method to outperform state-of-the-art methods.

The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS).

In this paper we report the set-up and results of the Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) organized in conjunction with the MICCAI 2012 and 2013 conferences. Twenty state-of-the-art tumor segmentation algorithms were applied to a set of 65 multi-contrast MR scans of low- and high-grade glioma patients-manually annotated by up to four raters-and to 65 comparable scans generated using tumor image simulation software. Quantitative evaluations revealed considerable disagreement between the human raters in segmenting various tumor sub-regions (Dice scores in the range 74%-85%), illustrating the difficulty of this task. We found that different algorithms worked best for different sub-regions (reaching performance comparable to human inter-rater variability), but that no single algorithm ranked in the top for all sub-regions simultaneously. Fusing several good algorithms using a hierarchical majority vote yielded segmentations that consistently ranked above all individual algorithms, indicating remaining opportunities for further methodological improvements. The BRATS image data and manual annotations continue to be publicly available through an online evaluation system as an ongoing benchmarking resource.

Partial volume estimation in brain MRI revisited.

We propose a fast algorithm to estimate brain tissue concentrations from conventional T1-weighted images based on a Bayesian maximum a posteriori formulation that extends the "mixel" model developed in the 90's. A key observation is the necessity to incorporate additional prior constraints to the "mixel" model for the estimation of plausible concentration maps. Experiments on the ADNI standardized dataset show that global and local brain atrophy measures from the proposed algorithm yield enhanced diagnosis testing value than with several widely used soft tissue labeling methods.

Flexible multivariate hemodynamics fMRI data analyses and simulations with PyHRF.

As part of fMRI data analysis, the pyhrf package provides a set of tools for addressing the two main issues involved in intra-subject fMRI data analysis: (1) the localization of cerebral regions that elicit evoked activity and (2) the estimation of activation dynamics also known as Hemodynamic Response Function (HRF) recovery. To tackle these two problems, pyhrf implements the Joint Detection-Estimation framework (JDE) which recovers parcel-level HRFs and embeds an adaptive spatio-temporal regularization scheme of activation maps. With respect to the sole detection issue (1), the classical voxelwise GLM procedure is also available through nipy, whereas Finite Impulse Response (FIR) and temporally regularized FIR models are concerned with HRF estimation (2) and are specifically implemented in pyhrf. Several parcellation tools are also integrated such as spatial and functional clustering. Parcellations may be used for spatial averaging prior to FIR/RFIR analysis or to specify the spatial support of the HRF estimates in the JDE approach. These analysis procedures can be applied either to volume-based data sets or to data projected onto the cortical surface. For validation purpose, this package is shipped with artificial and real fMRI data sets, which are used in this paper to compare the outcome of the different available approaches. The artificial fMRI data generator is also described to illustrate how to simulate different activation configurations, HRF shapes or nuisance components. To cope with the high computational needs for inference, pyhrf handles distributing computing by exploiting cluster units as well as multi-core machines. Finally, a dedicated viewer is presented, which handles n-dimensional images and provides suitable features to explore whole brain hemodynamics (time series, maps, ROI mask overlay).

Bayesian joint detection-estimation of cerebral vasoreactivity from ASL fMRI data.

Although the study of cerebral vasoreactivity using fMRI is mainly conducted through the BOLD fMRI modality, owing to its relatively high signal-to-noise ratio (SNR), ASL fMRI provides a more interpretable measure of cerebral vasoreactivity than BOLD fMRI. Still, ASL suffers from a low SNR and is hampered by a large amount of physiological noise. The current contribution aims at improving the recovery of the vasoreactive component from the ASL signal. To this end, a Bayesian hierarchical model is proposed, enabling the recovery of perfusion levels as well as fitting their dynamics. On a single-subject ASL real data set involving perfusion changes induced by hypercapnia, the approach is compared with a classical GLM-based analysis. A better goodness-of-fit is achieved, especially in the transitions between baseline and hypercapnia periods. Also, perfusion levels are recovered with higher sensitivity and show a better contrast between gray- and white matter.

Fast joint detection-estimation of evoked brain activity in event-related FMRI using a variational approach.

In standard within-subject analyses of event-related functional magnetic resonance imaging (fMRI) data, two steps are usually performed separately: detection of brain activity and estimation of the hemodynamic response. Because these two steps are inherently linked, we adopt the so-called region-based joint detection-estimation (JDE) framework that addresses this joint issue using a multivariate inference for detection and estimation. JDE is built by making use of a regional bilinear generative model of the BOLD response and constraining the parameter estimation by physiological priors using temporal and spatial information in a Markovian model. In contrast to previous works that use Markov Chain Monte Carlo (MCMC) techniques to sample the resulting intractable posterior distribution, we recast the JDE into a missing data framework and derive a variational expectation-maximization (VEM) algorithm for its inference. A variational approximation is used to approximate the Markovian model in the unsupervised spatially adaptive JDE inference, which allows automatic fine-tuning of spatial regularization parameters. It provides a new algorithm that exhibits interesting properties in terms of estimation error and computational cost compared to the previously used MCMC-based approach. Experiments on artificial and real data show that VEM-JDE is robust to model misspecification and provides computational gain while maintaining good performance in terms of activation detection and hemodynamic shape recovery.

Variational solution to the joint detection estimation of brain activity in fMRI.

We address the issue of jointly detecting brain activity and estimating underlying brain hemodynamics from functional MRI data. We adopt the so-called Joint Detection Estimation (JDE) framework that takes spatial dependencies between voxels into account. We recast the JDE into a missing data framework and derive a Variational Expectation-Maximization (VEM) algorithm for its inference. It follows a new algorithm that has interesting advantages over the previously used intensive simulation methods (Markov Chain Monte Carlo, MCMC): tests on artificial data show that the VEM-JDE is more robust to model mis-specification while additional tests on real data confirm that it achieves similar performance in much less computation time.